Comics on the Brain: Structure and Meaning in Sequential Image Comprehension

Just as syntax differentiates coherent sentences from scrambled word strings, sequential images must also use a cognitive system to distinguish coherent narratives from random strings of images. We conducted experiments analogous to two classic psycholinguistic studies to examine structure and semantics in sequential images. We compared Normal comic strips with both structure and meaning to sequences with Semantics Only, Structure Only, or randomly Scrambled panels. Experiment 1 used a target-monitoring paradigm, and found that RTs were slowest to target panels in Scrambled sequences, intermediate in Structural Only and Semantic Only sequences, and fastest in Normal sequences. Experiment 2 measured ERPs to the same strips. The largest N400 appeared in Scrambled and Structural Only sequences, intermediate in Semantic Only sequences and smallest in Normal sequences. Together, these findings suggest that sequential image comprehension is guided by an interaction between a structure and meaning, broadly analogous to syntax and semantics in language.